{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Deep Learning with TensorFlow\n",
    "\n",
    "In this notebook, we will go over the basics of Deep Learning using TensorFlow 2. TensorFlow is a framework which allows manipulations of tensors using constructs similar to numpy. In addition it has modules to allow auto differentiation to carryout back-propagation which forms the backbone of training a neural network.\n",
    "\n",
    "Prior to version 2, TensorFlow had two phases, first phase to build graph of computation and then in 2nd phase using `session` to pass data through the grapah and run back propagation. However with the introduction of eager execution in Tensorflow 2.X, it now has simplified a lot and the differences between PyTorch and TensorFlow have become very narrow. \n",
    "\n",
    "Ofcourse for production setup, you could still use the pre-complied graph to speed up things.\n",
    "\n",
    "## What are Neural Networks\n",
    "\n",
    "Deep learning is based on Artificial Neural networks which are made up of neurons. A neuron takes inputs, calculates the weighted sum and then passes the sum through some kind of non-linear function (called activation function) as shown below:\n",
    "\n",
    "![Neuron](./images/neuron.png \"Neuron\")\n",
    "\n",
    "<br/>\n",
    "<br/>\n",
    "\n",
    "We stack these neurons to make a neural network as shown below:\n",
    "![Neuron](./images/nn.svg \"Neuron\")\n",
    "\n",
    "Let us now create a neural network in PyTorch. We will use this network to train a model to take MNIST data as input and produce the class it belongs to.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MNIST \n",
    "\n",
    "MNIST dataset has images 28x28 pixels  = 784 pixels. \n",
    "\n",
    "We will have 10 units at the output layer to signify the digit (0-9) the image belongs to.\n",
    "\n",
    "Let us first load the data and print some images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "image size of `(60000,28,28)` means that we have 60000 images in the x_train dataset , with each image of size (28x28) (height x width)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1775c99e340>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAANaUlEQVR4nO3dfahc9Z3H8c9HTXxIgySG6DUN2gT/SBE2NSEErEuW0pL1n1gkixEWV4VboWKFhd1YJVWXQNjd7v4hWrj1IdmlGupDqZRKo6Gs2z8MieJqzKPKtXm4JrpBa8CQNX73j3tSrsmdMzdzzsyZe7/vFwwzc75zzvky3M89Z+acMz9HhABMfec13QCA3iDsQBKEHUiCsANJEHYgiQt6uTLbfPUPdFlEeLzplbbstlfa3mv7XdtrqywLQHe50+Psts+XtE/SdyUdlLRd0pqI2FUyD1t2oMu6sWVfJundiHg/Ik5K2ixpVYXlAeiiKmGfJ+nAmOcHi2lfYXvQ9g7bOyqsC0BFVb6gG29X4azd9IgYkjQksRsPNKnKlv2gpPljnn9d0uFq7QDoliph3y7pGtvfsD1d0i2SXqynLQB163g3PiK+sH23pN9JOl/SkxHxTm2dAahVx4feOloZn9mBruvKSTUAJg/CDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Lo6ZDN6I4lS5a0rN1yyy2l895xxx2l9V27Wo7TKUl67bXXSutl7r///tL6yZMnO142zsaWHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYBTXSaDd8eh169a1rE2fPr3udmqzevXq0vpzzz3Xo06mllajuFY6qcb2sKTPJJ2S9EVELK2yPADdU8cZdH8VER/XsBwAXcRndiCJqmEPSVtsv257cLwX2B60vcP2jorrAlBB1d346yPisO25kl62vSciXh37gogYkjQk8QUd0KRKW/aIOFzcH5X0K0nL6mgKQP06DrvtGbZnnn4s6XuSdtbVGIB6dXyc3fYCjW7NpdGPA09HxPo287Ab34E5c+aU1oeHh1vWZsyYUXM39Tlx4kRp/fbbby+tb968uc52pozaj7NHxPuS/qLjjgD0FIfegCQIO5AEYQeSIOxAEoQdSIJLXKeABx54oGWt7PJXSZo2bVpp/dNPPy2tX3rppaX1KtodWluzZk3X1j2ZtTr0xpYdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgOPsUd+jQodL6lVdeWVr/8MMPS+tXXHHFOfc0UYsWLSqt79mzp2vrnsw4zg4kR9iBJAg7kARhB5Ig7EAShB1IgrADSdQxsCP62EMPPVRab3e9+7x58+ps55xcfPHFja17KmLLDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcD17cu2Oo2/fvr20PjAwUGc7X7Ft27bS+vLly7u27sms4+vZbT9p+6jtnWOmzbb9su39xf2sOpsFUL+J7MZvlLTyjGlrJW2NiGskbS2eA+hjbcMeEa9KOnbG5FWSNhWPN0m6qd62ANSt03PjL4+IEUmKiBHbc1u90PagpMEO1wOgJl2/ECYihiQNSXxBBzSp00NvR2wPSFJxf7S+lgB0Q6dhf1HSbcXj2yT9up52AHRL2914289IWiFpju2Dkn4iaYOkX9q+U9IfJa3uZpPo3D333FNaX7p0aWm9m78L384rr7zS2LqnorZhj4hWI95/p+ZeAHQRp8sCSRB2IAnCDiRB2IEkCDuQBJe4TgKLFy8urW/ZsqVl7bLLLiud97zz+vf/PUM2d4Yhm4HkCDuQBGEHkiDsQBKEHUiCsANJEHYgCYZsngSWLFlSWp81q/WP+/bzcfR21q9fX1q/+eabe9TJ1DB5/xIAnBPCDiRB2IEkCDuQBGEHkiDsQBKEHUiC4+yTwBNPPFFav+qqq1rW7rvvvtJ5L7igf/8E2g0njXPDlh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkujfg6yYsHXr1rWs7d27t3TesmvhJ6LdcfoNGza0rF144YWV1o1z03bLbvtJ20dt7xwz7UHbh2y/Wdxu7G6bAKqayG78Rkkrx5n+7xGxuLj9tt62ANStbdgj4lVJx3rQC4AuqvIF3d223yp281t+8LM9aHuH7R0V1gWgok7D/jNJCyUtljQi6aetXhgRQxGxNCKWdrguADXoKOwRcSQiTkXEl5J+LmlZvW0BqFtHYbc9MObp9yXtbPVaAP2h7fjstp+RtELSHElHJP2keL5YUkgalvSDiBhpuzLGZ59y7HGHAv+zxx57rGXtrrvuKp332LHy74WXLSvfoXzvvfdK61NVq/HZ255UExFrxplc/msKAPoOp8sCSRB2IAnCDiRB2IEkCDuQBJe4opJ2l6m2O7xW5tSpU5Xq+Cq27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBMfZUcnjjz/etWU/9dRTpfXh4eGurXsqYssOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0m0/SnpWlc2iX9Keu7cuS1rL730Uum8GzduLK0/8sgjnbTUE/Pnzy+t79+/v7ReZVjmRYsWldb37NnT8bKnslY/Jc2WHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS4Hr2CXr22Wdb1q677rrSedsdqz5w4EBpfd++faX1Xbt2taytWLGidN5rr722tL527drSepXj6E8//XRp/YMPPuh42Thb2y277fm2f297t+13bP+omD7b9su29xf3s7rfLoBOTWQ3/gtJfx8RiyQtl/RD29+UtFbS1oi4RtLW4jmAPtU27BExEhFvFI8/k7Rb0jxJqyRtKl62SdJNXeoRQA3O6TO77aslfUvSNkmXR8SINPoPwfa4J4/bHpQ0WLFPABVNOOy2vybpeUn3RsSf7HHPtT9LRAxJGiqWMWkvhAEmuwkderM9TaNB/0VEvFBMPmJ7oKgPSDranRYB1KHtJa4e3YRvknQsIu4dM/1fJP1vRGywvVbS7Ij4hzbLmrRb9pUrV7asPfroo6XzLliwoNK6P/nkk9J62U8qt7tMtMqhM0lq9/fz0UcftawtXLiwdN7jx4931FN2rS5xnchu/PWS/lbS27bfLKb9WNIGSb+0faekP0paXUOfALqkbdgj4g+SWn1A/0697QDoFk6XBZIg7EAShB1IgrADSRB2IAl+SroG7S7V3L17d2n94YcfrrOdnvr8889L65dcckmPOsFp/JQ0kBxhB5Ig7EAShB1IgrADSRB2IAnCDiTBT0nX4NZbby2tX3TRRaX1mTNnVlr/8uXLW9ZuuOGGSss+ceJEab3sOn/0F7bsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE17MDUwzXswPJEXYgCcIOJEHYgSQIO5AEYQeSIOxAEm3Dbnu+7d/b3m37Hds/KqY/aPuQ7TeL243dbxdAp9qeVGN7QNJARLxhe6ak1yXdJOlvJB2PiH+d8Mo4qQboulYn1UxkfPYRSSPF489s75Y0r972AHTbOX1mt321pG9J2lZMutv2W7aftD2rxTyDtnfY3lGtVQBVTPjceNtfk/RfktZHxAu2L5f0saSQ9E8a3dW/o80y2I0HuqzVbvyEwm57mqTfSPpdRPzbOPWrJf0mIq5tsxzCDnRZxxfC2LakJyTtHhv04ou7074vaWfVJgF0z0S+jf+2pP+W9LakL4vJP5a0RtJije7GD0v6QfFlXtmy2LIDXVZpN74uhB3oPq5nB5Ij7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJNH2Bydr9rGkD8Y8n1NM60f92lu/9iXRW6fq7O2qVoWeXs9+1srtHRGxtLEGSvRrb/3al0RvnepVb+zGA0kQdiCJpsM+1PD6y/Rrb/3al0RvnepJb41+ZgfQO01v2QH0CGEHkmgk7LZX2t5r+13ba5vooRXbw7bfLoahbnR8umIMvaO2d46ZNtv2y7b3F/fjjrHXUG99MYx3yTDjjb53TQ9/3vPP7LbPl7RP0nclHZS0XdKaiNjV00ZasD0saWlENH4Chu2/lHRc0n+cHlrL9j9LOhYRG4p/lLMi4h/7pLcHdY7DeHept1bDjP+dGnzv6hz+vBNNbNmXSXo3It6PiJOSNkta1UAffS8iXpV07IzJqyRtKh5v0ugfS8+16K0vRMRIRLxRPP5M0ulhxht970r66okmwj5P0oExzw+qv8Z7D0lbbL9ue7DpZsZx+elhtor7uQ33c6a2w3j30hnDjPfNe9fJ8OdVNRH28Yam6afjf9dHxHWS/lrSD4vdVUzMzyQt1OgYgCOSftpkM8Uw489Lujci/tRkL2ON01dP3rcmwn5Q0vwxz78u6XADfYwrIg4X90cl/UqjHzv6yZHTI+gW90cb7ufPIuJIRJyKiC8l/VwNvnfFMOPPS/pFRLxQTG78vRuvr169b02Efbuka2x/w/Z0SbdIerGBPs5ie0bxxYlsz5D0PfXfUNQvSrqteHybpF832MtX9Msw3q2GGVfD713jw59HRM9vkm7U6Dfy70m6v4keWvS1QNL/FLd3mu5N0jMa3a37P43uEd0p6TJJWyXtL+5n91Fv/6nRob3f0miwBhrq7dsa/Wj4lqQ3i9uNTb93JX315H3jdFkgCc6gA5Ig7EAShB1IgrADSRB2IAnCDiRB2IEk/h/w6C81r/AB+wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let us plot one image\n",
    "plt.imshow(x_train[10].squeeze(), cmap='Greys_r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "flatten (Flatten)            (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 192)               150720    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 128)               24704     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 176,714\n",
      "Trainable params: 176,714\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(192, activation='relu'),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10)\n",
    "])\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forward pass and Calculate Cross Entropy Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to view the probability of classification of digit\n",
    "def view_classification(img, probs):\n",
    "    fig, (ax1, ax2) = plt.subplots(figsize=(6,7), ncols=2)\n",
    "    ax1.imshow(img)\n",
    "    ax1.axis('off')\n",
    "    ax2.barh(np.arange(10), probs)\n",
    "    ax2.set_aspect(0.1)\n",
    "    ax2.set_yticks(np.arange(10))\n",
    "    ax2.set_yticklabels(np.arange(10).astype(int), size='large');\n",
    "    ax2.set_title('Probability')\n",
    "    ax2.set_xlim(0, 1.1)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADECAYAAAA8lvKIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAThUlEQVR4nO3de5QcZZnH8e8vkxDIFUgAQwhEBAQBjRgvBJEswgbjYbksLkFAUJe4QWDdgyjLQRcFBVaPuoLoZiUYwBuuAVyIXBRFCYJOMAhoCJEkkAuXcAm5k0ye/aNrjs1U9dCTTFd11/w+5/TJ9FNvVT/p9HnyzlvVTykiMDOzfPQrOgEzs77ERdfMLEcuumZmOXLRNTPLkYuumVmOXHTNzHLkomvWx0gKSfts5b6LJR1VY9vhkh7PGivpIknf3bqMy6V/0QmYWX0kLQZ2AzqAtcBs4NyIWFNkXp0i4rfAm2ts+3Lnz5LGAouAARGxOZ/smodnumat5diIGAIcArwTuLh6oyRPpJqci65ZC4qIZcDPgYOS5YJPSnoCeAJA0lmSFkp6UdLPJO3e5RCTJT0paaWkr0jql+z3Jkn3SHoh2fZ9STt22fedkv4s6SVJ10naPtl3oqSlWflKukTSjcnT3yR/vixpjaQjkjwPrhq/q6T1knbZlvepGbnomrUgSWOAycAfk9DxwLuBt0g6Ergc+CdgFLAE+FGXQ5wAjKcyYz4O+FjnoZN9dwcOAMYAl3TZ91RgEvAmYD+6zLbr8L7kzx0jYkhE3Jvkd1rVmFOAX0TE8z08dtNz0TVrLbdIehm4D7gX6FwrvTwiXoyI9VSK4oyIeCgiNgL/DhyarKV2ujIZ/xTwDSpFjohYGBF3R8TGpOB9DTiiSw5XR8TTEfEi8KXOfbfRTODDnTNu4HTghl44btPx+o9Zazk+In5RHZAE8HRVaHfgoc4nEbFG0gvAaGBxEq4evyTZB0m7At8EDgeGUpmYvdQlh8x9t0VEPChpLXCEpBXAPsDPtvW4zcgzXbNyqG4XuBzYq/OJpMHACGBZ1ZgxVT/vmewDlaWFAN4aEcOo/MqvLq9Va9+tybXazOT1Tgf+NyI29PC4LcFF16x8fgB8VNI4SQOpLEE8GBGLq8ZcIGmnZG34X4EfJ/GhwBoqJ7lGAxdkHP+TkvaQtDNwUdW+9Xoe2ALs3SV+A5W15tOA63t4zJbhomtWMhHxS+BzwE+BFVROeE3pMuxWYC4wD7gduDaJf4HKybVVSXxWxkv8ALgLeDJ5XNbD/NZRWQueI+llSe9J4kupLIsE8NueHLOVyE3MzaxZSJoBLI+Inl4R0TJ8Is3MmkJydcWJwNsLTqWhvLxgZoWTdCnwKPCViFhUdD6N5OUFM7Mcdbu8cHS/D7kiW0PdveUnXS9HMis1Ly+YmeXIJ9KsTxo5cmSMHTu26DSspObOnbsyIjKb9bjoWp80duxY2tvbi07DSkrSklrbvLxgZpYjF10zsxy56JqZ5chF18wsRz6RZn3SI8tWMfbC22tuX3zFB3PMxvoSz3TNzHLkomtmliMXXWt5kg5I7mC7KrkD7glF52RWi4uutTRJ/ak05L4N2BmYCtwoab9CEzOrwUXXWt3+VG6M+PWI6IiIe4A5VO6zZdZ0XHSt1WV1KRNwUCooTZXULqm9Y92qxmdmlsFF11rdfOA5KjdaHCDp74EjgEFdB0bE9IgYHxHj2wYNzztPM8BF11pcRGwCjgc+CDwDnA/cBCwtMC2zmvzlCGt5EfEnKrNbACTdD8wsLiOz2jzTtZYn6a2Stpc0SNKngVHA9wpOyyyTi66VwenACipru+8Hjo6IjcWmZJbNywvW8iLiAuCCnuxz8OjhtLu/ghXAM10zsxx5pmt90ut1GWsUdy8zz3TNzHLkomstT9JYSbMlvSTpGUlXJz0ZzJqOi66VwTVUrlwYBYyjcs3u2UUmZFaLZwNV9I4DM+Nbtku/TcsmDk7FHjv3msz9N0XHtiXWA+9/9KTM+ODjVqRiWzZsaHQ6eXkjcHVEbACekXQHkP2PaVYwz3StDP4LmJJ8OWI08AHgjoJzMsvkomtlcC+Vme0rVHoutAO3dB3kLmPWDFx0raVJ6gfcCcwCBgMjgZ2AK7uOdZcxawYuutbqdgbGUFnT3RgRLwDXAZOLTcssm4uutbSIWAksAqZJ6i9pR+AM4OFCEzOrofRXL8Shb8uMP3HmdqnY14/8YebYAdqcih21w+pUbFNk/x+2hS3dpdir7j7opsz4uBs+loq9cdryzLEdK1/o1ZxycCLwDeCzQAfwK+DfikzIrJbSF10rv4iYB0zsyT5ueGNF8fKCmVmOXHTNzHLk5QXrk3qjy5g7htnWKH3RjctezIzP339WzpkUa96EGanYpHdntycYeHvLnUgzaxleXrCWJmlNl0eHpKuKzsusltLPdK3cImJI58+SBgPPAj8pLiOz7nmma2VyEpUWj78tOhGzWlx0rUzOAK6PiCg6EbNaXHStFCTtSaV5+cxuxrjLmBWu9Gu6y349JnvD/vUf43cbBqZiH5t9VnqgahygB/Ou9xyyIBW7buxd9R+g7/oIcF9ELKo1ICKmA9MBBo7a17NhK4RnulYWH6GbWa5Zs3DRtZYnaQIwGl+1YC3ARdfK4AxgVkSkW7+ZNZnSr+la+UXEJ3q6j7uMWVFKX3T3vKI9M37CTafUfQy9uikV23fRg1udU3deHjkiFfvFA0Mzx2b19K3lyEdOTsWG/eqxzLH5df8163u8vGBmlqPSz3TNstTTZcxdxKwRPNM1M8uRi66VgqQpkv4iaa2kv0o6vOiczLJ4ecFanqSjgSuBk4HfA6OKzcisttIX3dj0ama84/GFOWdSn2dP3C8VO3i7W2uMTn89uZbly3dOxYase7Lu/ZvcF4AvRsQDyfNlRSZj1h0vL1hLk9QGjAd2kbRQ0lJJV0vaIWOsG95Y4Vx0rdXtBgyg0kv3cGAc8Hbg4q4DI2J6RIyPiPFtg4bnmqRZJxdda3Xrkz+viogVEbES+BowucCczGpy0bWWFhEvAUvpUQNNs+KU/kRas3p+2qGZ8f1Pm5+K7dZW/wmzWg74TLrNbMc2H7VpXAecK+kOYBPwKeC2QjMyq8FF18rgUmAksADYANwEfKnQjMxqcNG1lhcRm4Czk0dd3GXMiuI1XTOzHLnompnlyMsL1ifV02WsFncfs23hotuLnjtnQmb8jGmzU7HThn01c+zQftttUw6XPn9IZjw2Zn8d2szy5eUFa3mSfi1pg6Q1yePxonMyq8VF18rinIgYkjzeXHQyZrW46JqZ5chF18rickkrJc2RNDFrgLuMWTMo/Ym0tgOzf9Nc8NGdUrEj3vvoNr3WbWOuyoxvyby/bv0nzBZu2pwZP/nb56die978bHYOq/9a9+u1oM8CfwZeBaYA/ydpXES85i8dEdOB6QADR+3rXg1WCM90reVFxIMRsToiNkbETGAO7jJmTcpF18ooABWdhFkWF11raZJ2lDRJ0vaS+ks6FXgfcGfRuZllKf2arpXeAOAyYH8q3SrnA8dHhK/VtaZUqqIbh41Lxc687ubMsccNXtmADBrzi8N5C0/OjI++8v5UrEQ9cusSEc8D7+zpfu4yZkXx8oKZWY5cdM3MclSq5QWzevWky5i7illv8kzXzCxHLrpWGpL2TbqN3Vh0Lma1lH55oa3Gnbn7NeD/mwFqy4xv2sYvnN5xQPYVGIef+slUbPj3H9i2F2tt3wL+UHQSZt3xTNdKQdIU4GXglwWnYtYtF11reZKGAV8E0h2AXjvOXcascC66VgaXAtdGxNPdDYqI6RExPiLGtw0anlNqZq9V+jVdKzdJ44CjgLcXnIpZXUpVdDVnXip27fHHZI698MwRqdied2bfvLFtfXY/223xxMcHZMbnH/PtXn+tkpsIjAWekgQwBGiT9JaIyL5Lp1mBSlV0rU+aDvyo6vmnqRThaYVkY/Y6XHStpUXEOmBd53NJa4ANSSMcs6bjomulEhGX1DPOXcasKL56wcwsR57pWp/UXcMbN7ixRip90e3484LM+N6fyTmRLg54YpfsDdkXW5hZSXh5wcwsRy661vIk3ShphaRXJC2Q9M9F52RWi4uulcHlwNiIGAb8A3CZpHcUnJNZJhdda3kR8VhEbOx8mjzeVGBKZjWV/kRas3r2xH2KTqFUJF0DnAnsAPwRmJ0xZiowFaBtWI0TmWYN5pmulUJEnA0MBQ4HZgEbM8a4y5gVzkXXSiMiOiLiPmAP3HvBmpSLrpVRf7yma03KRddamqRdJU2RNERSm6RJwCnAPUXnZpbFJ9Ks1QWVpYTvUJlELAE+FRG3FpqVWQ1NX3Q1cGAq9vKHsm8SsNOtj6ViW1av7vWcemrF+RNSsVvP+88ao9N/X6staeF4RE/3c5cxK4qXF8zMctT0M12zRuiuy1hX7jpmvckzXTOzHLnoWkuTNFDStZKWSFot6Y+SPlB0Xma1NM3ywoZj35UZH/7pp1Kxe/e5KnPsCX84JR18vDEn0vqPekMqtuykvTPH/vjcr6Ziu/ev/4TZsx2pL1cBMGB91H2MEusPPE3lZNpTwGTgJkkHR8TiIhMzy9I0Rddsa0TEWuCSqtBtkhYB7wAWF5GTWXe8vGClImk3YD8gff2gWRNw0bXSkDQA+D4wMyLmZ2yfKqldUnvHulX5J2iGi66VhKR+wA3Aq8A5WWPcZcyagdd0reVJEnAtsBswOSI2FZySWU1NU3QnfenezPj5Ix6t+xjzLxqWDq5599am1K0pE36Xit2ya/bF9lsYUPdxz1g8KRVbeN2bM8eOmJXOoY/6NnAAcFRErC86GbPueHnBWpqkvYBPAOOAZyStSR6nFpuZWbammemabY2IWAKop/u54Y0VxTNdM7McueiameWoVMsLfznqvwvOIPv/sN9tSH/l96wHP5I5dp+znkjFRqz1CbPe5i5jVhTPdM3McuSiay1P0jnJN802Svpe0fmYdadUywvWZy0HLgMmATsUnItZt1x0reVFxCwASeOBPQpOx6xbXl4wM8tR08x07znvsMz49Wenm5s/fNiMRqfzGje+MiYVW7Fpx1RsxkPZf4d9/qcjFdt7zrzMsVt6lJn1hKSpwFSAtmG7FJyN9VWe6Vqf4S5j1gxcdM3MctQ0ywtmW0tSfyqf5TagTdL2wOaI2FxsZmZpnulaGVwMrAcuBE5Lfr640IzMalBE7TvKHt3vQ4XfbrbfoEGp2NPnjcscO/MT30jFDtouuwHVkY+cnIqt+nX6Dr8Ae/14WSq2edGSzLHWM3dv+UmPO4T1hvHjx0d7e3sRL219gKS5ETE+a5tnumZmOXLRNTPLkU+kWZ/Uky5jtbj7mG0Nz3TNzHLkomstT9LOkm6WtFbSEkkfLjons1qafnlhy7p1qdjoK+7PHHvRFemvDNcyhCfrigH4Ys+m9y3gVSq3YB8H3C7p4Yh4rNCszDJ4pmstTdJg4B+Bz0XEmoi4D/gZcHqxmZllc9G1Vrcf0BERC6piDwMHdh0oaWrS7Ly9Y92q3BI0q+aia61uCNC1gq4ChnYd6IY31gxcdK3VrQGGdYkNA1YXkIvZ63LRtVa3AOgvad+q2NsAn0SzpuSiay0tItYCs4AvShos6TDgOOCGYjMzy+aia2VwNpUbUj4H/BCY5svFrFk1/XW6Zq8nIl4Eju/JPgePHk67v8ZrBfBM18wsRy66ZmY5ctE1M8uRi66ZWY5cdM3McuSia2aWI18yZn3S3Llz10h6vOg8gJHAyqKTSDiXtK3NY69aG7q9G7BZWUlqr3W31r6YBziXvPLw8oKZWY5cdM3McuSia33V9KITSDRLHuBcsvR6Hl7TNTPLkWe6ZmY5ctG1UpF0jKTHJS2UdGHGdkn6ZrL9T5IOqXffBuRyapLDnyTdL+ltVdsWS3pE0jxJ7Q3OY6KkVclrzZP0+Xr3bUAuF1Tl8aikDkk7J9t68z2ZIek5SY/W2N64z0lE+OFHKR5AG/BXYG9gOyo3qHxLlzGTgZ8DAt4DPFjvvg3IZQKwU/LzBzpzSZ4vBkbm9J5MBG7bmn17O5cu448F7unt9yQ51vuAQ4BHa2xv2OfEM10rk3cBCyPiyYh4FfgRlbtIVDsOuD4qHgB2lDSqzn17NZeIuD8iXkqePgDssQ2vt9V5NGjf3jjeKVSa0ve6iPgN8GI3Qxr2OXHRtTIZDTxd9XxpEqtnTD379nYu1T5OZWbVKYC7JM2VNDWHPA6V9LCkn0vqvH19Ye+JpEHAMcBPq8K99Z7Uo2GfE38N2MpEGbGul+fUGlPPvr2dS2Wg9HdUiu57q8KHRcRySbsCd0uan8zOGpHHQ8BeEbFG0mTgFmDfOvft7Vw6HQvMicpdQTr11ntSj4Z9TjzTtTJZCoyper4HsLzOMfXs29u5IOmtwHeB4yLihc54RCxP/nwOuJnKr7UNySMiXomINcnPs4EBkkbW+3fozVyqTKHL0kIvvif1aNznpDcWpf3woxkeVH5zexJ4I387yXFglzEf5LUnSH5f774NyGVPYCEwoUt8MDC06uf7gWMamMcb+Ns1++8Cnkren9zfk2TccCrrrYMb8Z5UHXMstU+kNexz4uUFK42I2CzpHOBOKmeZZ0TEY5L+Jdn+HWA2lTPTC4F1wEe727fBuXweGAFcIwlgc1Saq+wG3JzE+gM/iIg7GpjHScA0SZuB9cCUqFSYIt4TgBOAuyJibdXuvfaeAEj6IZWrNkZKWgr8BzCgKo+GfU78jTQzsxx5TdfMLEcuumZmOXLRNTPLkYuumVmOXHTNzHLkomtmliMXXTOzHLnompnl6P8BfgOuwMGSMTYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x504 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "img = x_train[7:8,:]\n",
    "logits = model(img)\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = tf.nn.softmax(logits).numpy()\n",
    "\n",
    "view_classification(img[0], prediction[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Back Propagation\n",
    "We need to now train the network to adjust its weights by first calculating Cross Entropy Loss and then back propagating the error to adjust weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create loss_fn\n",
    "loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train network\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=loss_fn,\n",
    "              metrics=['accuracy'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "1875/1875 [==============================] - 2s 975us/step - loss: 0.2140 - accuracy: 0.9358\n",
      "Epoch 2/2\n",
      "1875/1875 [==============================] - 2s 945us/step - loss: 0.0891 - accuracy: 0.9729\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1775cd61af0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train model\n",
    "model.fit(x_train, y_train, epochs=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 - 0s - loss: 0.1001 - accuracy: 0.9670\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.10011259466409683, 0.9670000076293945]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# evaluate model\n",
    "model.evaluate(x_test,  y_test, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADECAYAAAA8lvKIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAATKElEQVR4nO3de7hVdZ3H8ffHA964eAE0RRBNTVOL9HQRMxmzwehxvIyNmJpWIw2m1jxmOT7WOGmjTj3VpFnDJIbazSbURslLmZaYFhgmFiIJKBcVvCCHm3D4zh97nWl71trHDZy91t7rfF7Psx/O/u7fWufD9jxff/zWOr+tiMDMzPKxTdEBzMz6EjddM7McuemameXITdfMLEduumZmOXLTNTPLkZuuWR8jKSTtt4XHLpR0bI3XjpL0ZNZYSZdI+u6WJS6XfkUHMLP6SFoI7A50AquB6cD5EdFRZK4uEfEb4C01Xvv3rq8ljQIWAP0jYmM+6ZqHZ7pmreX4iBgIHAa8E7i0+kVJnkg1OTddsxYUEUuAnwOHJMsFn5L0FPAUgKRzJM2X9JKkn0nas9spxkt6WtIKSV+RtE1y3Jsl3SfpxeS170vaudux75T0J0kvS7pB0vbJsWMlLc7KK+kySTcnT3+d/PmKpA5JRyc5D60av5uktZKGbc371IzcdM1akKQRwHjgD0npRODdwFslHQNcCfwDsAewCPhRt1OcBLRTmTGfAHy869TJsXsCBwEjgMu6HXs6MA54M3AA3WbbdXhf8ufOETEwIh5I8p1RNeY04BcRsXwzz9303HTNWsttkl4BHgQeALrWSq+MiJciYi2VpjglIh6NiPXAvwBHJGupXa5Oxj8DfINKkyMi5kfEvRGxPml4XwOO7pbh2oh4NiJeAr7cdexWmgp8pGvGDZwJ3NQL5206Xv8xay0nRsQvqguSAJ6tKu0JPNr1JCI6JL0IDAcWJuXq8YuSY5C0G/BN4ChgEJWJ2cvdMmQeuzUi4hFJq4GjJS0D9gN+trXnbUae6ZqVQ/V2gUuBvbueSBoADAGWVI0ZUfX1yOQYqCwtBPC2iBhM5Z/86va9ah27JVmrTU2+35nA/0TEus08b0tw0zUrnx8AH5M0WtJ2VJYgHomIhVVjLpK0S7I2/Gngx0l9ENBB5SLXcOCijPN/StJeknYFLqk6tl7LgU3Avt3qN1FZaz4DuHEzz9ky3HTNSiYifgl8AfgpsIzKBa8J3YbdDswCZgN3Atcn9X+jcnFtZVKflvEtfgDcAzydPK7YzHxrqKwFz5D0iqT3JPXFVJZFAvjN5pyzlcibmJtZs5A0BVgaEZt7R0TL8IU0M2sKyd0VJwPvKDhKQ3l5wcwKJ+lyYA7wlYhYUHSeRvLygplZjnpcXvjANh92R7aGunfTT7rfjmRWal5eMDPLkS+kWZ80dOjQGDVqVNExrKRmzZq1IiIyN+tx07U+adSoUcycObPoGFZSkhbVes3LC2ZmOXLTNTPLkZuumVmO3HTNzHLkpmtmliM3XTOzHLnpmpnlyE3XWp6kg5JPsF2ZfALuSUVnMqvFTddamqR+VDbkvgPYFZgI3CzpgEKDmdXgpmut7kAqH4z49YjojIj7gBlUPmfLrOm46Vqry9qlTMAhqaI0UdJMSTOXL1/e+GRmGdx0rdXNBV6g8kGL/SX9LXA0sGP3gRExOSLaI6J92LDMvUjMGs5N11paRGwATgQ+BDwHXAjcAiwuMJZZTd5lzFpeRPyRyuwWAEkPAVOLS2RWm2e61vIkvU3S9pJ2lPRZYA/gewXHMsvkpmtlcCawjMra7vuBD0TE+mIjmWXz8oK1vIi4CLio6Bxm9fBM18wsR266ZmY5ctM1M8uRm661PEmjJE2X9LKk5yRdm+zJYNZ03HStDK6jcufCHsBoKvfsnltkILNaPBuoosMPzqxv2jb9Ni0ZOyBVe+L86zKP3xCdWxdsM7x/zimZ9QEnLEvVNq1b1+g4edkHuDYi1gHPSboLyP6PaVYwz3StDP4TmJD8csRw4IPAXQVnMsvkpmtl8ACVme2rVPZcmAnc1n2QdxmzZuCmay1N0jbA3cA0YAAwFNgFuLr7WO8yZs3ATdda3a7ACCpruusj4kXgBmB8sbHMsrnpWkuLiBXAAmCSpH6SdgbOAh4rNJhZDaW/eyGOeHtm/amzt03Vvn7MDzPH9tfGVO3YHValahsi+/9hm9jUU8Rede8ht2TWR9/08VRtn0lLM8d2rnixVzPl4GTgG8DngU7gV8A/FxnIrJbSN10rv4iYDYwtOIZZXby8YGaWIzddM7McuemameWo9Gu6ccVLmfW5B07LOUmxZo+ZkqqNe3f29gTb3dlyF9LMWoZnutbSJHV0e3RKuqboXGa1lH6ma+UWEQO7vpY0AHge+Elxicx65pmulckpVLZ4/E3RQcxqcdO1MjkLuDEiouggZrW46VopSBpJZfPyqT2M8S5jVrjSr+kuuX9E9gsH1n+O367bLlX7+PRz0gNV4wSbMe96z2HzUrUbRt1T/wn6ro8CD0bEgloDImIyMBmgvb3ds2ErhGe6VhYfpYdZrlmzcNO1lidpDDAc37VgLcBN18rgLGBaRKS3fjNrMqVf07Xyi4hPFp3BrF6lb7ojr5qZWT/pltPqPode25Cq7b/gkS3O1JNXhg5J1X7x8KDMsVl7+tZyzOOnpmqDf/VE5tj8dv8163u8vGBmliM3XTOzHLnpmpnlyE3XSkHSBEl/lrRa0l8kHVV0JrMspb+QZuUn6QPA1cCpwO+APYpNZFZb6ZtubHgts9755Pyck9Tn+ZMPSNUO3fb2GqPTv55cy9Klu6ZqA9c8XffxTe7fgC9FxMPJ8yVFhjHriZcXrKVJagPagWGS5ktaLOlaSTtkjPWGN1Y4N11rdbsD/anspXsUMBp4B3Bp94ERMTki2iOifdiwYbmGNOvipmutbm3y5zURsSwiVgBfA8YXmMmsJjdda2kR8TKwmM3aQNOsOKW/kNaslk86IrN+4BlzU7Xd2+q/YFbLQZ9LbzPbudVnbRo3AOdLugvYAHwGuKPQRGY1uOlaGVwODAXmAeuAW4AvF5rIrAY3XWt5EbEBODd5mDU1r+mameXITdfMLEduumZmOfKabi964bwxmfWzJk1P1c4Y/NXMsYO22XarMly+/LDMeqzP/nVoM8uXZ7rW8iTdL2mdpI7k8WTRmcxqcdO1sjgvIgYmj7cUHcasFjddM7McuelaWVwpaYWkGZLGZg3wLmPWDEp/Ia3t4Ox/ac772C6p2tHvnbNV3+uOEddk1jdlfr5u/RfM5m/YmFk/9dsXpmojb30+O8Oqv9T9/VrQ54E/Aa8BE4D/lTQ6Il73l46IycBkgPb2du/VYIXwTNdaXkQ8EhGrImJ9REwFZuBdxqxJuelaGQWgokOYZXHTtZYmaWdJ4yRtL6mfpNOB9wF3F53NLEvp13St9PoDVwAHUtmtci5wYkT4Xl1rSqVqunHk6FTt7BtuzRx7woAVDUjQmH84XDD/1Mz68KsfStVKtEduXSJiOfDOonOY1cvLC2ZmOXLTNTPLkZuumVmO3HTNzHLkpmulIWn/ZLexm4vOYlZLqe5eyNJW45O5t2nA/2/6qy2zvmErf+H0roOy78A46vRPpWo7ff/hrftmre1bwO+LDmHWE890rRQkTQBeAX5ZcBSzHrnpWsuTNBj4EpDeAej147zLmBXOTdfK4HLg+oh4tqdBETE5Itojon3YsGE5RTN7vdKv6Vq5SRoNHAu8o+AoZnUpVdPVjNmp2vUnHpc59uKzh6RqI+/O/vDGtrXZ+9lujac+0T+zPve4b/f69yq5scAo4BlJAAOBNklvjYjsT+k0K1Cpmq71SZOBH1U9/yyVJjypkDRmb8BN11paRKwB1nQ9l9QBrEs2wjFrOm66VioRcVnRGcx64rsXzMxy5Jmu9UmPL1nJqIvvLDqGNbmFV32o189Z+qbb+ad5mfV9P5dzkG4OeqrGfaLZN1uYWUl4ecHMLEduutbyJN0saZmkVyXNk/SPRWcyq8VN18rgSmBURAwG/g64QtLhBWcyy+Smay0vIp6IiPVdT5PHmwuMZFZT6S+kNavnT96v6AilIuk64GxgB+APwPSMMROBiQBtg73hjRXDM10rhYg4FxgEHAVMA9ZnjPn/Xcbadtwp74hmgJuulUhEdEbEg8BeeO8Fa1JuulZG/fCarjUpN11raZJ2kzRB0kBJbZLGAacB9xWdzSyLL6RZqwsqSwnfoTKJWAR8JiJuLzSVWQ1N33S13Xap2isfzv6QgF1ufyJV27RqVa9n2lzLLhyTqt1+wX/UGJ3++1ptyRaOR2/ucYcO34mZDfi9erM34uUFM7McuemameXITdfMLEduutbSJG0n6XpJiyStkvQHSR8sOpdZLU1zIW3d8e/KrO/02WdStQf2uyZz7Em/Py1dfLIxF9L67fGmVG3JKftmjv3x+V9N1fbsV/8Fs+c7U79cBUD/tVH3OUqsH/AslYtpzwDjgVskHRoRC4sMZpalaZqu2ZaIiNXAZVWlOyQtAA4HFhaRyawnXl6wUpG0O3AAkL5/0KwJuOlaaUjqD3wfmBoRczNenyhppqSZy5f7E9qtGG66VgqStgFuAl4DzssaU73L2LBh3trRiuE1XWt5kgRcD+wOjI+IDQVHMqupaZruuC8/kFm/cMicus8x95LB6WLHu7c0Uo8mjPltqnbbbtkf6b2J/nWf96yF41K1+Te8JXPskGnpDH3Ut4GDgGMjYm3RYcx64uUFa2mS9gY+CYwGnpPUkTxOLzaZWbammemabYmIWASo6Bxm9fJM18wsR266ZmY5KtXywp+P/a+CE2T/P+y369K/8nvOIx/NHLvfOU+lakNW+4KZWVl4pmtmliM3XWt5ks5LftNsvaTvFZ3HrCelWl6wPmspcAUwDtih4CxmPXLTtZYXEdMAJLUDexUcx6xHXl4wM8tR08x077vgyMz6jeemNzd/7MgpjY7zOje/OiJVW7Zh51RtyqPZf4f9/rszVdt3xuzMsZs2K5ltDkkTgYkAI0eOLDiN9VWe6Vqf4V3GrBm46ZqZ5ahplhfMtpSkflR+ltuANknbAxsjYmOxyczSPNO1MrgUWAtcDJyRfH1poYnMamiamW7b/Y9m1vf53Y6p2uEXfDpz7NRPfiNVO2Tb7A2ojnn81FRt5f3pT/gF2PvHS1K1jQsWpWr7MyvzeGusiLiM1384pVnT8kzXzCxHbrpmZjly0zUzy5GbrplZjtx0reVJ2lXSrZJWS1ok6SNFZzKrpWnuXqhl05o1qdrwqx7KHHvJVelfGa5lIE/XVQPwzZ5N71vAa1Q+gn00cKekxyLiiUJTmWXwTNdamqQBwN8DX4iIjoh4EPgZcGaxycyyuelaqzsA6IyIeVW1x4CDuw+UNDHZ7Hzm8uXLcwtoVs1N11rdQGBlt9pKYFD3gd7wxpqBm661ug5gcLfaYGBVAVnM3pCbrrW6eUA/SftX1d4O+CKaNSU3XWtpEbEamAZ8SdIASUcCJwA3FZvMLJubrpXBuVQ+kPIF4IfAJN8uZs2q6e/TNXsjEfEScGLROczq4ZmumVmO3HTNzHLkpmtmliM3XTOzHLnpmpnlyE3XzCxHvmXM+qRZs2Z1SHqy6BzAUGBF0SESzpK2pTn2rvWCImLL45i1KEkzI6LdOf7KWfLJ4eUFM7McuemameXITdf6qslFB0g0Sw5wliy9nsNrumZmOfJM18wsR266ViqSjpP0pKT5ki7OeF2Svpm8/kdJh9V7bAOynJ5k+KOkhyS9veq1hZIelzRb0swG5xgraWXyvWZL+mK9xzYgy0VVOeZI6pS0a/Jab74nUyS9IGlOjdcb93MSEX74UYoH0Ab8BdgX2JbKB1S+tduY8cDPAQHvAR6p99gGZBkD7JJ8/cGuLMnzhcDQnN6TscAdW3Jsb2fpNv544L7efk+Sc70POAyYU+P1hv2ceKZrZfIuYH5EPB0RrwE/ovIpEtVOAG6MioeBnSXtUeexvZolIh6KiJeTpw8De23F99viHA06tjfOdxqVTel7XUT8GniphyEN+zlx07UyGQ48W/V8cVKrZ0w9x/Z2lmqfoDKz6hLAPZJmSZqYQ44jJD0m6eeSuj6+vrD3RNKOwHHAT6vKvfWe1KNhPyf+NWArE2XUut+eU2tMPcf2dpbKQOlvqDTd91aVj4yIpZJ2A+6VNDeZnTUix6PA3hHRIWk8cBuwf53H9naWLscDM6LyqSBdeus9qUfDfk4807UyWQyMqHq+F7C0zjH1HNvbWZD0NuC7wAkR8WJXPSKWJn++ANxK5Z+1DckREa9GREfy9XSgv6Sh9f4dejNLlQl0W1roxfekHo37OemNRWk//GiGB5V/uT0N7MNfL3Ic3G3Mh3j9BZLf1XtsA7KMBOYDY7rVBwCDqr5+CDiugTnexF/v2X8X8Ezy/uT+niTjdqKy3jqgEe9J1TlHUftCWsN+Try8YKURERslnQfcTeUq85SIeELSPyWvfweYTuXK9HxgDfCxno5tcJYvAkOA6yQBbIzK5iq7A7cmtX7ADyLirgbmOAWYJGkjsBaYEJUOU8R7AnAScE9ErK46vNfeEwBJP6Ry18ZQSYuBfwX6V+Vo2M+JfyPNzCxHXtM1M8uRm66ZWY7cdM3McuSma2aWIzddM7McuemameXITdfMLEduumZmOfo/zqVkwsE38L4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x504 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check the prediction on same image as the one used before training\n",
    "logits = model(img)\n",
    "\n",
    "# Predict the class from the network output\n",
    "prediction = tf.nn.softmax(logits).numpy()\n",
    "\n",
    "view_classification(img[0], prediction[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that model is able to correctly predict the digit as 3 after training while before training it predicting all digits with almost equal probability i.e. it was randomly predicting the digit."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
